Fine-Tuning in a Design Experiment 
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Quek, Tay, Toh, Leong, and Dindyal (2011) proposed that a design-theory-practice troika 
should always be considered for a designed package to be acceptable to the research users 
who, in this case, are teachers and schools. This paper describes the fine-tuning to the 
MProSE problem-solving design made by the teachers in the school after first round of 
teaching. This process involved teacher input from their experience, and detailed time- 
consuming discussions and learning between the researcher-designers and the teacher- 
implementers. 

A mainstream school participated in our research project Mathematical Problem 
Solving for Everyone, (MProSE) (Toh, Quek, Leong, Dindyal & Tay, 2011). As part of the 
initialization phase of MProSE, the research team conducted a ten-hour professional 
development course for the mathematics teachers to use the MProSE design to teach 
mathematical problem solving. The Head of Department (HOD) of Mathematics, with the 
endorsement of the principal, coordinated and oversaw the implementation of the problem- 
solving module for four classes of Year 7 students (130 in total) over Term 1 and early 
Term 2 in 2012. The ten one-hour lessons were taught within the allocated curriculum time 
by four teachers, one to each class. These four teachers had different years of teaching 
experience. The HOD and these four teachers worked closely with the researchers. 

The design experiment approach used in MProSE posits that a design-theory-practice 
troika should always be considered for a designed package to be acceptable to the final 
research users, who in this case are the teachers and the school (Quek, Tay, Toh, Leong, & 
Dindyal, 2011). The design could be fine-tuned for improvement based on the feedback by 
the implementers. However, the modifications due to the fine-tuning should stay within 
the parameters specified by the theory for the design. In the sections following we describe 
two rounds of fine-tuning. In the first round of fine-tuning, which took place while the 
module was being taught, the teacher-implementers were most concerned with the types of 
problems that were most suitable for their students. As such, the focus of the fine-tuning 
efforts was centered around discussion of problems — changing the problems as well as 
having deeper understanding of existing problems. In the second round of fine-tuning, 
which is in anticipation of the next run of the module, we also looked at other aspects of 
implementation, which include the overall scheme of work, the structure of lesson, the 
activities to develop positive attitudes in the students, and scaffolding measures. 

First Round of Fine-tuning 

Good and appropriate mathematical task design is a crucial component in teaching of 
the MProSE problem-solving lesson. A well-designed task is essential to developing 
students’ abilities and confidence in handling unfamiliar problem situations. In designing 
the tasks for the first round of implementation of the MProSE design, the research team 
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proposed a set of 17 problems. These problems were crafted based on s tudents’ 
mathematical competency (as reported by their teachers). The 17 problem-solving tasks 
were to be used in the ten lessons. A problem could be used as the Problem of the Day 
(POD), which is to be attempted by students in class, or as a Homework (HW) problem. 
For each task, the designer-researchers provided the problem-solving heuristics and 
scaffolding hints, which they derived from Polya’s and Schoenfeld’s work. The students 
attempted the task by using the Practical Worksheet. This worksheet, which is a key 
feature of the MProSE design, is described in Toh, Quek, Leong, Dindyal and Tay (pp. 21- 
24, 2011). The 17 problems, though prepared in advance by the designer-researchers, were 
not meant to be “cast-in-stone” but could be fine-tuned to lit the implementers’ needs and 
constraints. 

There were three modes of line-tuning (a) discussion by teacher-implementers by 
themselves before each lesson was conducted, (b) discussion immediately after the 
MProSE lesson with designer-researchers, and (c) discussion via emails between HOD and 
designer-researchers. The examples that follow illustrate tweaks that result from each of 
these modes respectively. 

The MProSE teacher-implementers held a pre-lesson discussion, which were not 
attended by the designer-researchers, to talk about how to solve the POD and to line -tune 
the problem tasks and teaching approaches. The teachers reported that these pre-lesson 
discussions helped them to clarify doubts, build confidence, fine-tune the teaching 
strategies proposed by the designer-researchers and feel better prepared to facilitate the 
MProSE problem-solving lessons. For example, the teachers knew that their students had 
not learnt factorization by Tenn 1 to attempt the proposed POD for Lesson 2: 

111 1 

Simplify 1 1 K..-I . leaving your answer in the simplest form in 

1x2 2x3 3x4 100x101 

terms of n. Justify your answer. 

As a result of their discussion, this POD was replaced by the problem: Find the last 
digit of 13 77 . 

The designer-researchers would sit in different classes to observe the lesson and take 
field notes. A post-lesson discussion with the designer-researchers was held after each 
lesson to exchange perspectives and suggest further fine-tuning. The teachers would give 
views on their own teaching (e.g., hard for them to provide adequate scaffolding) and 
surface any observations they made (e.g., the students reported not liking to fill up the 
Worksheet). 

We report here about an example of fine-tuning that resulted from an extended 
discussion. Lesson 5 focused on the Check and Expand stage. The teachers were not aware 
that the expansion to a problem could be linked to the structure of the solution to the 
original problem which in this case is: 

HW5: The figure is a 7 x 7 array where each cell is a square. Find the number of squares contained 



Even though it was suggested to them as a possible extension — Find the number of 
rectangles in an m x ri rectangular array — the teachers did not consider it because they 
could not see the link in the solution. The designer-researchers walked them through the 
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solution by referencing the original solution to the number of squares, through step-by-step 
questioning. The following is a recall of the incident. 

(Designer-researcher) DR : How did you solve the number-of-squares problem? 

(Teacher-implementer) TI : By considering cases. 

DR : Would you like to do the same for the number-of-rectangles problem? 

TI : I don’t see how it could be done. . .there are too many cases to consider. 

DR : Just try. . .start with some cases . . . 

TI : Let me see... is 1 by 1 a rectangle? I think so. There are 49. What’s next ... 1 by 
2 ... there are 7x6 rectangles ... (starts recording on apiece of paper)... Now 1 
by 3 ... 7x5 rectangles ... Oh, I see a pattern now!! ... (finally completes the 
solution, looking overjoyed and beaming with confidence). 

If we had known this, we would have guided the students along similarly! 

One teacher-implementer was candid enough to admit a lack of mathematical ability 
and knowledge. The post-lesson discussion seemed important for the teachers to 
understand the solution and the problem at a deeper level. 

Typically a pre- and post lesson discussion among teachers would take an hour each. 
Thus sustaining these weekly discussions required dedication from the teacher- 
implementers and support from the school-leaders (Principal and HOD) in terms of time 
for discussion (as a professional learning community), curricular adaptations, and words of 
encouragement. 

The last mode of discussion for fine-tuning the design was via emails. The HOD would 
email his concerns anytime to the research team. For example, in one email, the HOD 
sought advice from the designer-researchers on the choice of a HW question (see Figure 1 
below). 

2. We are also concern with the homework problem. Can we replace it with the average speed 
problem that we have proposed please ? (See homework slide). 

We hope that you would be able to advise us. We will be conducting the first run on Wednesday 
0745-0845 with 1 A and the second run on Thu 1415 - 1515. 

Y our advice, please 
Thank you. 
with regards, 

Name removed 

Figure 1. An excerpt of email from the HOD to seek advice 

The original HW8 problem was on the “Intersection of Two Squares” (for details of 
problem, refer to Schoenfeld, 1985). After discussion, the problem was changed to: 

Jeremy and James wondered which car has the highest average speed. Each of the cars, Ferrari and 
Maserati were to make two runs and the average speed will be measured. 

The Ferrari ran two lengths of 1000 m in 10 sec and 8.5 sec respectively. 

The Maserati completed two runs of 10 sec, covering 1000 m and 1175m respectively. 

Jeremy went on to calculate the speed for each run and took the average of the two speeds for each 
car. He found that the Ferrari was the faster car but James thinks that Jeremy made some mistakes in 
his calculations. Was Jeremy’s calculation correct? When will average speed be the average of two 
speeds? Why? 

As a result of these three modes of discussion, three proposed problems were changed 
and one not used. 
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Fine-Tuning for the Second Round 

After the first run of the module, the teachers were generally positive about the 
outcome and had decided to carry it out again for the next cohort of Year 7 students. In 
between the two implementations, we intend to enter into intense discussions on the second 
round of fine-tuning. At the time of writing, two sessions had been conducted. Each 
session took about two hours. The discussions were in such great detail that Teacher S 
remarked that “the fine-tuning process in determining whether a problem is suitable for the 
students or not is not that clear-cut and [is] time-consuming”. He added that “thinking how 
to help students in class and explain clearly to the students is rather challenging and the 
different types of problems we need to discuss benefit me”. We acknowledge that the fine- 
tuning process is resource-intensive. In fact, we think it needs to be necessarily so in order 
to bring about deep understanding of changes through the teacher development process. 

We state briefly here the changes that were made for the next round. First, they 
evaluated the suitability of having the MProSE lessons conducted in the “Supplementary 
lessons” slot — what was done in the first run, after usual lesson hours in the afternoon. The 
school had since decided that MProSE is such a vital piece in their mathematics 
instructional programme that they would carry it out during normal lesson times. Next, to 
better equip the Year 7 students with more cognitive resources for problem solving, the 
module will be taught in Term 2 instead of Term 1 and end in middle of Term 3. This is to 
allow students to have opportunities to accumulate relevant content knowledge in the first 
tenn. Teachers found that this is beneficial also from the perspective of having more time 
for preparations. A new suggestion put forward by the researchers was the use of colour 
scaffold stickers to differentiate the different levels of cognitive scaffold that a teacher may 
need to give to the students. They agreed to try out this innovation. [In fact, the scaffolding 
cards were tested in another MProSE school and the preliminary results are reported in 
another paper in this symposium]. There were also more discussions about suitability of 
problems. For example, the “Phoney Russian Roulette” problem was deemed inappropriate 
due to students’ lack of pre-requisite knowledge of probability, and was replaced by the 
problem of “finding the sum of all the digits from 1 to 99”. 

Conclusion 

In this paper, we report the support given to teachers as we fine-tune various aspects of 
the MProSE design. It is resource-intensive, and no less than the “hard and unglamorous 
work” that Schoenfeld (1985) envisioned. But from the learning and changes that we 
observed, we are encouraged to know that it is also necessary and far-sighted work. In 
order to cater to the specific needs of students to improve their problem-solving ability and 
to empower the teacher-implementers, the fine-tuning of the MProSE design is integral to a 
sustainable and effective implementation of the problem solving module. The iterative 
nature of the design experiment methodology has made the participating teacher- 
implementers aware that it takes time and effort to understand the learning needs of the 
students. 
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